NVIDIA has open-sourced two new large models, Nemotron-4-Minitron-4B and Nemotron-4-Minitron-8B, which utilize structured pruning and knowledge distillation for efficient training. This significantly reduces training requirements, minimizing data and computing power consumption. Compared to traditional methods, the new models reduce training token data by 40 times and save 1.8 times in computing costs. By optimizing Llama-3.18B, structured pruning simplifies the model structure, while knowledge distillation enhances performance.

The team from Peking University and the Hong Kong University of Science and Technology made a big splash with a new training method that has achieved GPT-4 level performance with an 8B-sized medical expert model. This is no small feat, and they have also introduced a new concept, "stability gap," to explain certain phenomena observed during the continuous pre-training of large language models.Image Source Note: The image is generated by AI, and the image is provided by Midjourney, an image autho

The research team has proposed a novel training method to enhance the image generation capabilities of DALL-E3. This method integrates synthetic and real titles to address spatial perception and text presentation issues. Advanced language models like GPT-4 play a key role in improving text quality and depth. The research shows significant improvements in the quality and accuracy of DALL-E3's image generation. This strategy lays the foundation for the continued development of future text-to-image generation technologies.